Add distributionally robust optimization (DRO) for fairness explainer - #569
Open
propcgamer20-png wants to merge 2 commits into
Open
Add distributionally robust optimization (DRO) for fairness explainer#569propcgamer20-png wants to merge 2 commits into
propcgamer20-png wants to merge 2 commits into
Conversation
New explainers/maxmin-fairness.md. Covers the fairness objective that
minimizes the maximum group-level loss (Rawls' difference principle)
rather than equalizing a rate across groups, and the real conflict
between it and parity metrics: a model can move toward max-min fairness
while still failing demographic parity.
- Objective-function contrast: demographic parity's constrained average
loss vs max-min's min-of-max over per-group losses.
- Concrete example on the Audit 03 German Credit data (age, under 30): a
plain logistic-regression baseline (older 81.0% acc, younger 70.9%,
worst-group error 29.1%) vs an iterative group-reweighting max-min loop
(worst-group error 27.9%, older unchanged, overall accuracy flat) that
leaves the ~12-point selection-rate gap almost untouched. Cross-refs the
repo's own frozen accuracy_equality_diff (-0.153, p=0.031) as the gap
max-min targets and S1-S4 do not.
- Fully runnable detection/implementation code (fit_maxmin + maxmin_report),
deterministic output verified against the pasted block.
- Limitations: tiny/noisy subgroups without a min-group-size floor;
worst-off is only defined once a loss is chosen; silent on between-group
gaps; levelling-down permitted at the margin without a Pareto constraint.
- Citations verified: Rawls (1971); Hashimoto et al. ICML 2018; Martinez,
Bertran & Sapiro ICML 2020; Diana et al. AIES 2021.
Regenerated the HTML page, OG images, explainers-data.{js,json}, package
mirror, sitemap.xml and llms-full.txt; check_explainer_count.py --fix
bumped the 60 -> 61 mentions. The companion DRO explainer (yakew7#544) will add
the reciprocal cross-links.
…closes yakew7#544) New explainers/distributionally-robust-optimization.md. Covers DRO as an in-processing fairness technique - minimise worst-case loss over an uncertainty set of subgroup distributions - and contrasts it with this repo's S3 constraint-based ExponentiatedGradient (needs a chosen parity metric and a named group up front; DRO needs neither, and no group labels at prediction time). - Min-max objective spelled out against ExponentiatedGradient's constrained objective; the online group-DRO algorithm (Sagawa et al. 2020) and the uncertainty-set radius as the 'robustness knob'. - Concrete example on Audit 03 German Credit (age): an ERM baseline (young held-out log_loss 0.588) vs online group DRO sweeping the robustness knob. DRO reliably drives q(young) from 0.50 to ~0.63, but worst-group test log_loss does not improve (drifts to ~0.599) and overall accuracy stays flat - a direct, honest illustration of Sagawa et al.'s 'importance of regularization for worst-case generalization'. Cross-refs the frozen S0 -> S3 demographic_parity_diff (-0.129 -> +0.023) to show DRO targets a different quantity than the constraint does. - Runnable fit_group_dro + dro_report; pasted output verified against the extracted block, deterministic with pinned libs. - Limitations: uncertainty-set size is a free parameter with no data-driven default; still needs training-time group labels; worst-case train loss != worst-case test loss; grouping/loss choice bounds it. - Citations verified: Sagawa et al. ICLR 2020; Hashimoto et al. ICML 2018; Duchi & Namkoong AoS 2021; Agarwal et al. ICML 2018. Also adds the reciprocal cross-links in maxmin-fairness.md (yakew7#543). Regenerated HTML, OG images, explainers-data.{js,json}, package mirror, sitemap.xml, llms-full.txt; check_explainer_count.py --fix bumped 61 -> 62. Stacked on the yakew7#543 branch since the two explainers cross-reference each other; retarget to main once yakew7#543 merges.
propcgamer20-png
requested review from
Shreyash0712 and
yakew7
as code owners
September 10, 2026 09:58
Contributor
|
@propcgamer20-png is attempting to deploy a commit to the yashkewlani2020-gmailcom's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
@yakew7 @Shreyash0712 - new PR touching a path you own, please review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #544.
New
explainers/distributionally-robust-optimization.mdon DRO as an in-processing fairness technique: minimize worst-case loss over an uncertainty set of subgroup distributions, rather than constraining a named parity metric.Structure
Min-max objective spelled out against
ExponentiatedGradient's constrained objective (this repo's S3). DRO needs neither a chosen fairness metric nor a named group up front, and no group labels at prediction time.Concrete example - Audit 03 German Credit,
age:log_lossDRO reliably finds and reweights toward the worst group, but on a capacity-limited linear model it buys no worst-group generalization - a direct, honest illustration of Sagawa et al.'s headline ("On the Importance of Regularization for Worst-Case Generalization"). Cross-references the frozen S0 -> S3
demographic_parity_diff(-0.129 -> +0.023) to show DRO targets a different quantity than the constraint does.Runnable code -
fit_group_dro(online group-DRO loop) anddro_report. Pasted Script Execution Output verified byte-for-byte against the extracted block; deterministic with pinned libs.Limitations - the uncertainty-set size is a free parameter with no data-driven default; still needs training-time group labels; worst-case train loss is not worst-case test loss; the grouping and loss choice bound what it can see.
Citations (verified against proceedings)
Sagawa, Koh, Hashimoto & Liang, ICLR 2020 (arXiv:1911.08731); Hashimoto et al., ICML 2018 (PMLR 80:1929-1938); Duchi & Namkoong, Annals of Statistics 49(3) 2021; Agarwal et al., ICML 2018.
check_em_dash.py,check_broken_links.py(1570 links),check_explainer_count.py(62),check_generated_files_current.py,tests/test_build_explainers.pyall pass.